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ABSTRACT 

A system and method of adding hyperlinked information to a television broadcast. 
The broadcast material is analyzed and one or more regions within a frame are identified. 
Additional information can be associated with a region, and can be transmitted in 
encoded form, using timing information to identify the frame with which the information 
is associated. The system comprising a video source and an encoder that produces a 
transport stream in communication with the video source, an annotation source, a data 
packet stream generator that produces encoded annotation data packets in communication 
with the annotation source and the encoder, and a multiplexer system in communication 
with the encoder and the data packet stream generator. The encoder provides timestamp 
information to the data packet stream generator and the data packet stream generator 
2 0 synchronizes annotation data from the annotation source with a video signal from the 

video source in response to the timestamp information. The multiplexer generates a 
digital broadcast signal that includes an augmented transport stream from the transport 
2 5 stream and the encoded data packets. A receiver displays the annotation information 

associated with the video signal in response to a viewer request on a frame by frame 
basis. A viewer can respond interactively to the material, including performing 
commercial transactions, by using a backchannel that is provided for interactive 
communication. 
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